Dialect separation assessment using log-likelihood score distributions

Dialect differences within a given language represent major challenges for sustained speech system performance. For speech recognition, little if any knowledge exists on differences between dialects (e.g. vocabulary, grammar, prosody, etc.). Effective dialect classification can contribute to improved ASR, speaker ID, and spoken document retrieval. This study, presents an approach to establish a metric to estimate the separation between dialects, and to provide some sense of expected speech system performance. The proposed approach compares dialects based on their loglikelihood score distributions. From the score distributions, a numerical measure is obtained to assess the separation between resulting GMM dialect models. The proposed scheme is evaluated on a corpus of Arabic dialects. The sensitivity of the dialect separation score is also quantified based on controlled mixing of dialect data for the case of measuring dialect training data purity. The resulting scheme is shown to be effective in measuring dialect distance, and represents an important objective way of assessing dialect differences within a common language.